Summary

• add a new distributed checkpoint/restart reproducibility test in test/test_distributed_utils.jl
• mirror the serial checkpointer workflow: run to iteration 3, checkpoint, restore into a fresh simulation, then continue to iteration 6
• assert field agreement using the same tolerances as the serial test (T,S,h: rtol=1e-13; u,v,ui,vi: rtol=1e-10), and assert exact clock time/iteration equality

Why this is needed in addition to serial tests

• serial checkpoint tests validate restart consistency for single-process execution, but they do not exercise MPI domain decomposition, collective reductions, or halo-exchange ordering
• distributed runs can introduce additional floating-point drift modes that do not appear in serial, so we need explicit multi-rank coverage to keep reproducibility-within-tolerance guarantees from regressing

Implementation notes

• test uses an MPI-aware Distributed(CPU(), partition=Partition(mpi_ranks, 1))
• local checkpoint files are verified and cleaned up per rank
• test is skipped with a warning when only one MPI rank is active

with Codex